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Abstract. This study focuses on the use of emerging technologies such as Artificial 
Intelligence (AI) smart speakers and smartphone applications for improving the 
English language skills of L1 Japanese undergraduates. An empirical investigation 
was carried out with 82 Japanese students. Participants were required to study a 
variety of online English programmes using AI speakers over an eight-month 
period. The results showed that students using AI speakers outperformed on the 
Test of English for International Communication (TOEIC) a group of non-AI users, 
who instead exclusively used online materials. This research suggests integrating 
blended learning, including AI and Virtual Reality (VR), may be an effective way to 
improve the English proficiency of native Japanese. 
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1. Introduction 


The present study focuses on the use of emerging technologies such as smart 
speakers and smartphone applications for the purpose of improving the English 
language skills of undergraduates whose L1 is Japanese. Two case studies were 
carried out with the goal of exploring the implementation of a blended learning 
English language programme incorporating the AI speakers Google Home Mini 
and Amazon Alexa to improve the English proficiency of undergraduate Japanese 
students. The programme also aimed at fostering the development of intercultural 
awareness and at ascertaining how students felt about using AI and VR to study 
English. 
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2. Method 
2.1. Participants 


The two case studies were conducted over two semesters (April 2019 to January 
2020). A total of 82 undergraduates (L1 Japanese) participated. Case Study 1 
participants were divided into two groups: Group | (n=30) used an AI speaker 
during the blended learning training programme (April 2019 to January 2020), 
and Group 2 (n=29) did not use an AI speaker during the same training period. 
Case Study 2 included a total of 23 students. The participants were divided into 
six subgroups. 


2.2. Training procedure 


The following technologies were utilised: Amazon Alexa* and Google Home Mini° 
(see Figure 1), as well as ATR CALL Brix’, and the social networking service 
(Facebook, Twitter, and Line), and other programmes. In Case Study 1, TOEIC 
was used to determine if participants’ English skills improved and ascertain the 
effectiveness of the Al/blended learning programme. TOEIC was administered to 
Groups | and 2 as a pretest in April 2019 and posttest in January 2020. 


Figure 1. Amazon Alexa and Google Home Mini 


4. https://www.amazon.com/Amazon-Echo-And-Alexa-Devices/b?ie=UTF8&node=98 18047011 

5. https://rithvikvibhu.github.io/GHLocalApi/#top 

6. ATR CALL Grix is a sort of e-learning materials in which students could study TOEIC, vocabulary, listening, and reading: 
https://school.uchida.co.jp/index.cfm/23,5784,74,292,html 
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During training, Alexa was utilised by Group | participants who filmed and wrote 
in journals about their experiences. The AI speaker was integrated into their daily 
lives (e.g. for assistance in cooking, weather reports, movie recommendations, and 
so on) over the ten-month period. They each set a timer while interacting with the 
AI speaker to practise English listening, speaking, and vocabulary skills (could you 
explain how?). 


Group 2 participants used only ATR CALL Brix online materials at home, which 
focused on TOEIC and vocabulary learning. Thus, the main difference in training 
between groups was whether either the AI speaker or ATR CALL Brix was used, 
although the utilisation of other online activities (e.g. studying about World 
Heritage sites) were identical for both groups. 


Case Study 2 participants were administered both TOEIC and the Oral Proficiency 
Interview by Computer (OPIC) test in April 2019 and again in January 2020. The 
AI speakers were integrated into their daily lives over the ten-month period. A timer 
was set at home while interacting with the AI speaker to practise English listening, 
speaking, and vocabulary skills along with other various software programs. Half 
of the participants used Google Home Mini to improve English listening and 
speaking skills using the applications Best Teacher, Travel English, Let’s play 
around with English, and BBC/CNN news. The other half also used Home Mini to 
improve their listening and vocabulary skills, along with the programmes Kikutan, 
English Quiz by Arc, Liberty English, and Kindle. 


All participants of Case Study 2 also used VR goggles to interact within a variety of 
authentic environments. While studying with the AI speakers, participants recorded 
short movie clips of their experiences which were uploaded to a Facebook group 
page so the videos could be easily viewed by all participants. The participants 
also kept written journals with observations about the contents and duration of 
their studies, periodically recording their thoughts using a smartphone. At the 
end of training, participants of all six subgroups delivered presentations of their 
impressions of the Al/blended learning lesson training with smart speakers, with a 
majority indicating it had a positive effect on their learning experience. 


Case Study 2 included the following training tasks: 
* practiced English using AI speakers; 


¢ watched online TED talks using mobile devices, wrote 300-word 
summaries, created PowerPoint presentations, and discussed summaries 
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with a group of English native speakers four times over an eight-month 
period; 


¢ studied worldviews — a collection of attitudes, values, stories, and beliefs 
that influence our every thought and action about the world around us 
(Gray, 2011) — after viewing online lectures by several Oxford University 
scholars, and they also delivered PowerPoint presentations and created 
digital stories with iPads and presented their summaries to a group of 
English native speakers and discussed comparative worldviews; and 


¢ interacted with English L1 speakers (8 American university undergraduates) 
who evaluated their presentations and discussed worldviews and cultural 
issues. 


3. Results and discussion 
3.1. TOEIC and OPIC results 


In Case Study 1, mean TOEIC scores of Group | improved from 407 (SD:113) to 
604 (SD:92), an increase of 197 points. Mean TOEIC scores of Group 2 improved 
from 447 (SD:93) to 598 (SD:147), an increase of 147 points. 


In Case Study 2, mean TOEIC scores improved pretest to posttest from 461 (SD:136) 
to 681 (SD:141), an increase of 229 points. Mean OPIC speaking test scores also 
improved pretest to posttest from 3.9 (SD:0.9) to 4.7 (SD:1.25), respectively. The 
pre/posttest TOEIC results in both case studies were analysed using a series of 
t-tests, indicating the differences were statistically significant (p<.01). 


3.2. Post-training survey 


Post-training surveys were administered to all participants at the end of their 
respective Al/blended learning training to ascertain overall impressions of the 
programme. Responses to a few questions are summarised below: 


¢ AlIspeaker was useful in improving my English skills: 84% agreed (n=47). 
¢ Al speaker was useful in improving listening skills: 87% agreed (n=23). 
¢ Al speaker was useful in improving speaking skills: 57% agreed (n=23). 
¢ Al speaker was useful in improving reading skills: 13% agreed (n=23). 

¢ Al speaker was useful in improving writing skills: 4.3% agreed (n=23). 
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¢ Presentation practice with PowerPoint helped improve English proficiency: 
100% agreed (n=47). 


In both case studies, participants’ TOEIC scores considerably improved during the 
training period. However, the participants who used an AI speaker outperformed 
the non-AI speaker group in Case Study 1. Additionally, in Case Study 2, the 
integration of AI speakers and blended learning helped the participants improve 
their TOEIC scores by a mean of 229 points, while participants who did not use the 
AI speaker improved by roughly 150 points, although last year’s group improved 
by a mean score of more than 200 points. 


Survey results showed 84% of participants agreed the AI speaker was helpful in 
improving their English, particularly listening skills. Fifty-seven percent agreed 
the AI speaker was useful in improving speaking skills. The OPIC speaking test 
results with a mean increase from 3.9 (SD:0.9) to 4.7 (SD:1.25) would appear to 
verify these results. Conversely, only a small percentage of students agreed the AI 
speaker was helpful in improving reading and writing skills. 


We acknowledge the limitation of our study in accurately assessing the efficacy 
of AI smart speakers exclusively in developing L2 skills since a variety of 
supplemental online programmes and activities were utilised as part of the 
Al/blended learning learning programme. What is most important in L2 learning 
is face-to-face interaction and how much comprehensible input is gained from 
various learning sources, whether it be analogue or digital environments (Polat, 
2016). The distinction between classroom and outside-class activities was clearly 
drawn so each student could acquire as much comprehensible input as possible 
and be engaged in interactions and discussions using critical and creative thinking. 


4. Conclusions 


In line with Obari and Lambacher (2019), both TOEIC and OPIC speaking test and 
survey results revealed a combination of AI/blended learning lessons had a positive 
effect on the native Japanese students overall English language learning. Both their 
listening and oral skills improved, which may have been due to the integration 
of the language learning activities which concentrated on a social constructivist 
approach while utilising AI smart speakers. These results would appear to suggest 
that the integration of blended learning along with 21st century skills, including 
AI and VR, may be an effective way to improve the English proficiency of native 
Japanese adult learners. 
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